Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning

Intrinsically motivated spontaneous exploration is a key enabler of autonomous lifelong learning in human children. It enables the discovery and acquisition of large repertoires of skills through self-generation, self-selection, self-ordering and self-experimentation of learning goals. We present an algorithmic approach called Intrinsically Motivated Goal Exploration Processes (IMGEP) to enable similar properties of autonomous or self-supervised learning in machines. The IMGEP algorithmic architecture relies on several principles: 1) self-generation of goals, generalized as fitness functions; 2) selection of goals based on intrinsic rewards; 3) exploration with incremental goal-parameterized policy search and exploitation of the gathered data with a batch learning algorithm; 4) systematic reuse of information acquired when targeting a goal for improving towards other goals. We present a particularly efficient form of IMGEP, called Modular Population-Based IMGEP, that uses a population-based policy and an object-centered modularity in goals and mutations. We provide several implementations of this architecture and demonstrate their ability to automatically generate a learning curriculum within several experimental setups including a real humanoid robot that can explore multiple spaces of goals with several hundred continuous dimensions. While no particular target goal is provided to the system, this curriculum allows the discovery of skills that act as stepping stone for learning more complex skills, e.g. nested tool use. We show that learning diverse spaces of goals with intrinsic motivations is more efficient for learning complex skills than only trying to directly learn these complex skills

Impact of Robot Initiative on Human-Robot Collaboration

International audienceThis paper presents a study on the impact of autonomy in the context of human-robot collaboration. We consider two conditions: i) a semi-autonomous robot that decides when to execute a supporting action, and ii) a support robot that has to be instructed of each action on a collaborative task. The semi-autonomous robot gradually learns how to support the human through experience. We found that users prefer the semi-autonomous robot and that the behavior was closer to their expectations despite them being more afraid of it. We also found that even if users noticed the robot was learning in one case, they wanted more autonomy in both conditions

Autonomous exploration, active learning and human guidance with open-source Poppy humanoid robot platform and Explauto library

International audienceOur demonstration presents an open-source hardware and software platform which allows non-roboticistsresearchers to conduct machine learning experiments to benchmark algorithms for autonomous explorationand active learning. In particular, in addition to showing the general properties of the platform such asits modularity and usability, we will demonstrate the online functioning of a particular algorithm whichallows efficient learning of multiple forward and inverse models and can leverage information from humanguidance. A first aspect of our demonstration is to illustrate the ease of use of the 3D printed low-costPoppy humanoid robotic platform, that allows non-roboticists to quickly set up and program roboticexperiments. A second aspect is to show how the Explauto library allows systematic comparison andevaluation of active learning and exploration algorithms in sensorimotor spaces, through a Python API toselect already implemented exploration algorithms. The third idea is to showcase Active Model Babbling,an efficient exploration algorithm dynamically choosing which task/goal space to explore and particulargoals to reach, and integrating social guidance from humans in real time to drive exploration towardsparticular objects or actions.[Forestier and Oudeyer, 2016] Forestier, S. and Oudeyer, P.-Y. (2016). Modular active curiosity-driven discovery oftool use. In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea.[Lapeyre et al., 2014] Lapeyre, M., Rouanet, P., Grizou, J., Nguyen, S., Depraetre, F., Le Falher, A., and Oudeyer,P.-Y. (2014). Poppy Project: Open-Source Fabrication of 3D Printed Humanoid Robot for Science, Educationand Art. In Digital Intelligence 2014, page 6, Nantes, France.[Moulin-Frier et al., 2014] Moulin-Frier, C., Rouanet, P., Oudeyer, P.-Y., and others (2014). Explauto: an open-source Python library to study autonomous exploration in developmental robotics. In ICDL-Epirob-InternationalConference on Development and Learning, Epirob

Postural Optimization for an Ergonomic Human-Robot Interaction

International audienceIn human-robot collaboration the robot's behavior impacts the worker's safety, comfort and acceptance of the robotic system. In this paper we address the problem of how to improve the worker's posture during human-robot collaboration. Using postural assessment techniques, and a personalized human kinematic model, we optimize the model body posture to fulfill a task while avoiding uncomfortable or unsafe postures. We then derive a robotic behavior that leads the worker towards that improved posture. We validate our approach in an experiment involving a joint task with 39 human subjects and a Baxter torso-humanoid robot

Temporal Segmentation of Pair-Wise Interaction Phases in Sequential Manipulation Demonstrations

International audienceWe consider the problem of learning from complex sequential demonstrations. We propose to analyze demonstrations in terms of the concurrent interaction phases which arise between pairs of involved bodies (hand-object and object-object). These interaction phases are the key to decompose a full demonstration into its atomic manipulation actions and to extract their respective consequences. In particular, one may assume that the goal of each interaction phase is to achieve specific geometric constraints between objects. This generalizes previous Learning from Demonstration approaches by considering not just the motion of the end-effector but also the relational properties of the objects' motion. We present a linear-chain Conditional Random Field model to detect the pair-wise interaction phases and extract the geometric constraints that are established in the environment, which represent a high-level task oriented description of the demonstrated manipulation. We test our system on single- and multi-agent demonstrations of assembly tasks, respectively of a wooden toolbox and a plastic chair

Robot Programming from Demonstration, Feedback and Transfer

International audienceThis paper presents a novel approach for robot instruction for assembly tasks. We consider that robot programming can be made more efficient, precise and intuitive if we leverage the advantages of complementary approaches such as learning from demonstration, learning from feedback and knowledge transfer. Starting from low-level demonstrations of assembly tasks, the system is able to extract a high-level relational plan of the task. A graphical user interface (GUI) allows then the user to iteratively correct the acquired knowledge by refining high-level plans, and low-level geometrical knowledge of the task. This combination leads to a faster programming phase, more precise than just demonstrations, and more intuitive than just through a GUI. A final process allows to reuse high-level task knowledge for similar tasks in a transfer learning fashion. Finally we present a user study illustrating the advantages of this approach

A Multimodal Dataset for Object Model Learning from Natural Human-Robot Interaction

International audienceLearning object models in the wild from natural human interactions is an essential ability for robots to perform general tasks. In this paper we present a robocentric multimodal dataset addressing this key challenge. Our dataset focuses on interactions where the user teaches new objects to the robot in various ways. It contains synchronized recordings of visual (3 cameras) and audio data which provide a challenging evaluation framework for different tasks. Additionally, we present an end-to-end system that learns object models using object patches extracted from the recorded natural interactions. Our proposed pipeline follows these steps: (a) recognizing the interaction type, (b) detecting the object that the interaction is focusing on, and (c) learning the models from the extracted data. Our main contribution lies in the steps towards identifying the target object patches of the images. We demonstrate the advantages of combining language and visual features for the interaction recognition and use multiple views to improve the object modelling. Our experimental results show that our dataset is challenging due to occlusions and domain change with respect to typical object learning frameworks. The performance of common out-of-the-box classifiers trained on our data is low. We demonstrate that our algorithm outperforms such baselines

A Multimodal Dataset for Interactive and Incremental Learning of Object Models

This work presents an incremental object learning framework oriented to human-robot assistance and interaction. To learn new object models from interactions with a human user, the robot needs to be able to perform multiple recognition tasks: (a) recognize the type of interaction, (b) segment regions of interest from acquired data, and (c) learn and recognize object models. The contributions on this work are focused on the recognition modules of this human-robot interactive framework. First, we illustrate the advantages of multimodal data over camera-only datasets. We present an approach that recognizes the user interaction by combining simple image and language features. Second, we propose an incremental approach to learn visual object models, which is shown to achieve comparable performance to a typical offline-trained system. We utilize two public datasets, one of them presented and released in this work. This dataset contains synchronized recordings from user speech and three cameras mounted on a robot, which captured the user teaching object names to the robot

A Multimodal Human-Robot Interaction Dataset

International audienceThis works presents a multimodal dataset for Human-Robot Interactive Learning. 1 The dataset contains synchronized recordings of several human users, from a stereo 2 microphone and three cameras mounted on the robot. The focus of the dataset is 3 incremental object learning, oriented to human-robot assistance and interaction. To 4 learn new object models from interactions with a human user, the robot needs to 5 be able to perform multiple tasks: (a) recognize the type of interaction (pointing, 6 showing or speaking), (b) segment regions of interest from acquired data (hands and 7 objects), and (c) learn and recognize object models. We illustrate the advantages 8 of multimodal data over camera-only datasets by presenting an approach that 9 recognizes the user interaction by combining simple image and language features

Relational Activity Processes for Modeling Concurrent Cooperation

International audienceIn human-robot collaboration, multi-agent domains , or single-robot manipulation with multiple end-effectors, the activities of the involved parties are naturally concurrent. Such domains are also naturally relational as they involve objects, multiple agents, and models should generalize over objects and agents. We propose a novel formalization of relational concurrent activity processes that allows us to transfer methods from standard relational MDPs, such as Monte-Carlo planning and learning from demonstration, to concurrent cooperation domains. We formally compare the formulation to previous propositional models of concurrent decision making and demonstrate planning and learning from demonstration methods on a real-world human-robot assembly task